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INTRODUCTION 


Rehabilitation  of  Visual  and  Perceptual  Dysfunction  after  Severe  Traumatic  Brain  Injury 

The  aim  of  our  project  is  to  conduct  preliminary  evaluations  of  new  rehabilitation  strategies  and  new  functional 
assessment  methods  for  homonymous  hemianopia  (HH)  and  spatial  neglect  (SN),  disabling  visual  and  cognitive 
perception  conditions  that  commonly  occur  as  a  result  of  severe  traumatic  brain  injury  (TBI)  and  stroke.  Both 
HH  and  SN  prevent  detection  of  objects  on  the  affected  side,  resulting  in  unsafe  walking  and  driving.  In  our 
pilot  study  we  are  evaluating  a  novel  optical  device  combined  with  a  new  computerized  perceptual-motor 
training  regimen  in  helping  people  with  HH  and  SN  detect  and  avoid  obstacles  on  the  affected  side.  The  optical 
device,  expansion  prism  (EP)  glasses,  uses  high  power  prism  segments  embedded  in  a  regular  spectacle  lens  to 
project  areas  from  the  affected  (blind/neglected)  side  onto  the  unaffected  (seeing)  side  while  leaving  central 
vision  uninterrupted.  The  purpose  of  the  perceptual-motor  training  is  to  help  people  learn  how  to  interpret  the 
information  from  the  peripheral  prisms  so  that  they  can  correctly  identify  the  location  of  objects  detected  via  the 
prisms  and  respond  appropriately  (e.g.,  avoid  a  collision  when  walking  or  turn  to  face  a  person  approaching  on 
the  blind/neglected  side).  The  effects  of  the  EP  glasses  and  training  is  being  evaluated  using  realistic  tasks 
representative  of  everyday  mobility  challenges  including  detection  of  pedestrian  hazards  in  a  driving  simulator 
and  obstacle  collision  judgments  in  a  virtual  mall.  In  addition,  this  project  addresses  another  important  basic 
research  question;  namely,  the  effect  of  the  interventions  on  eye  movement  behaviors.  Eye  movements  are 
being  measured  without  and  with  EP  glasses  and  after  training  in  the  driving  simulator,  the  collision  judgment 
task  and  a  natural  walking  task.  The  eye  movement  data  will  provide  a  basic  understanding  of  how  participants 
are  using  the  device  and  will  help  guide  future  developments  of  prismatic  devices  for  HH  and  SN. 


BODY 

I.  Task  1  -  Preparation  of  study  protocols. 

This  task  was  completed  in  year  1 . 


II.  Task  2  -  Complete  development  of  the  perceptual-motor  training  program 

This  task  was  mostly  conducted  in  year  1.  However,  throughout  year  2  we  have  continued  to  add  additional 
enhancements  to  the  program,  fixes  to  previously  hidden  bugs,  and  modifications  to  allow  greater  flexibility  and 
ease  of  use,  and  improve  the  more  complex  aspects  of  the  training.  This  task  is  now  finished. 

Accomplishments 

•  The  program  now  includes  five  levels  of  training  and  has  been  used  extensively  for  training  sessions  in  the 
main  pilot  study  (Task  5). 

•  We  have  implemented  animated  user  feedback  screens,  a  template  task  for  each  level  of  training  that 
simplifies  setting  up  new  tasks,  the  use  of  video  as  a  fixation  target  and  multiple  default  prism  zones  per 
subject  with  integrity  checking. 

•  In  addition,  we  have  extended  the  training  to  more  real-world  situations  by  incorporating  detection  of 
natural  hazards  in  natural  driving  videos. 

III.  Task  3  -  Functional  assessments  in  the  driving  simulator  and  virtual  mall 

In  year  1  we  completed  the  modification  and  testing  of  existing  scenarios  in  the  driving  simulator  (detection  of 
potential  pedestrian  hazards)  and  the  existing  collision  judgment  task  in  the  virtual-mall  walking  simulator.  In 
year  2,  both  the  driving  simulator  and  virtual  mall  tasks  have  been  successfully  deployed  in  the  main  pilot 
study.  Our  efforts  under  this  task  in  year  2  have  mainly  focused  on  the  eye  and  head  tracking  aspects,  as  well  as 
the  development  of  data  analysis  programs. 

Accomplishments  -  Driving  simulator 

•  We  have  modified  existing  data  analysis  programs  for  analyzing  detection  performance  in  the  driving 
simulator,  including  detection  rates  and  reaction  times,  and  are  in  the  process  of  adding  the  capability  to 
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determine  the  proportion  of  pedestrian  appearances  for  which  there  would  have  been  a  collision  in  the  real 
world  (missed  detections  and  very  late  responses). 

•  We  are  also  in  the  process  of  modifying  existing  programs  to  evaluate  lane  position  and  steering. 

•  We  have  streamlined  the  driving  simulator  data  analysis  process  and  have  added  capability  to  the  data 
analysis  program  to  produce  customized,  publication  quality  plots  of  gaze. 

•  We  have  devoted  much  effort  to  implementing  calibration  tests  and  verification  procedures  for  the 
SmartEye  eye  and  head  tracking  system  that  we  are  using  to  record  eye  and  head  movements  when  driving 
with  and  without  prism  glasses  so  that  we  can  determine  how  participants  are  using  the  EP  glasses  and  the 
effects  of  perceptual-motor  training  on  eye  movement  behaviors. 

•  We  have  also  been  working  on  developing  an  algorithm  to  quantify  eye  and  head  movement  behaviors  in 
the  driving  simulator.  The  algorithm  has  been  applied  to  the  data  from  one  participant  and  now  needs  to  be 
run  on  the  complete  data  set. 

Accomplishments  -  Virtual  mall  walking  simulator 

•  We  have  added  new  features  to  the  existing  collision  judgment  program  including  “catch  trials”  (trials 
without  a  target),  a  calibration  feature,  and  an  option  for  manually  tagging  data  when  a  fixation  loss  occurs. 

•  We  have  been  developing  data  consolidation  programs  and  also  data  analysis  programs,  in  particular  for 
collision  judgment  responses  that  cannot  be  fit  with  a  cumulative  Gaussian. 

•  As  participants  perform  some  of  the  collision  judgments  with  fixed  gaze,  we  have  developed  a  program  for 
monitoring  fixation.  The  first  implementation  was  with  an  old  remote  IR  eye  tracker  (ISCAN),  which  we 
subsequently  replaced  with  a  SmartEye  head  and  eye  tracker  released  from  another  experimental  set  up. 

IV.  Task  4  -  Outdoor  walking  task 

We  have  made  much  progress  on  this  task  in  the  last  12  months.  At  the  beginning  of  year  2  we  purchased  a 
PositiveScience  eye  tracker  and  associated  software  and  by  the  end  of  the  first  quarter  had  implemented  it  in  the 
main  pilot  study.  Eye  and  head  movements  are  now  being  routinely  recorded  while  patients  walk  a  short 
outdoor  route. 

Accomplishments 

•  We  completed  setting  up  the  Positive  Science  eye  tracker  and  two  separate  motion  sensors  for  recording  of 
head  position  relative  to  the  body  when  walking,  developed  calibration  procedures,  finalized  two  outdoor 
walking  routes,  received  training  from  an  orientation  and  mobility  specialist,  and  then  started  to  collect  eye 
and  head  position  data  from  all  participants  in  the  pilot  study. 

•  We  received  a  new  computer  from  Schepens  (at  no  cost)  with  which  we  developed  a  system  for  backing-up, 
processing,  and  analyzing  the  approximately  150  GB  per  month  of  eye  and  head  movement  data  collected. 

•  We  have  been  processing  eye  and  head  movement  data,  and  are  currently  developing  a  method  of 
integrating  them  (not  a  trivial  task)  so  that  we  can  analyze  scanning  behaviors  based  on  gaze  direction. 

V.  Task  5-  Pilot  study  EP  glasses  and  training 

Our  main  focus  in  year  2  has  been  on  recruiting  and  running  participants  in  the  pilot  study.  We  have  made  good 
progress  despite  one  research  assistant  leaving  and  another  starting  in  the  second  quarter  of  the  year.  The  whole 
study  involves  at  least  15  visits  per  participant.  This  year  we  have  run  219  patient  visits,  a  total  contact  time  of 
about  550  hours.  Our  original  statement  of  work  was  for  2  years,  but  we  have  been  granted  a  no  cost  extension 
for  an  additional  year  so  that  we  can  complete  data  collection  for  the  main  pilot  study. 

Enrollment 

The  aim  is  to  have  12  patients  with  HH  without  SN  and  12  patients  with  HH  with  SN  complete  the  pilot  study. 
Allowing  for  attrition,  a  total  of  15  patients  with  HH  without  SN  and  15  patients  with  HH  with  SN  were 
approved  for  participation. 
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In  the  last  12  months  we  have  prioritized  recruitment  of  patients  with  HH  without  SN  in  order  to  reach  our  goal 
of  12  patients  to  complete  the  whole  study.  We  have  also  established  routes  of  referral  for  patients  with  HH 
with  SN  (Dr  Houston  will  refer  patients  from  his  clinic  at  Spaulding  Rehabilitation  Hospital).  We  will  focus  on 
recruitment  of  patients  with  SN  when  the  cohort  of  patients  without  SN  has  completed  all  training  visits.  (Due 
to  the  large  number  of  visits,  we  can  only  reasonably  accommodate  two  patients  in  training  at  the  same  time). 

HH  without  SN 

•  In  total,  since  the  start  of  the  study,  22  patients  with  HH  without  SN  have  been  screened  for  the  main  pilot 
study  of  which  13  met  the  criteria  and  have  been  enrolled  in  the  study;  5  have  completed  the  protocol,  7  are 
part  way  through  the  protocol,  1  withdrew  and  0  are  currently  lost  to  follow  up. 

•  Of  these,  12  were  enrolled  within  the  last  12  months. 

HH  with  SN 

•  In  total,  since  the  start  of  the  study,  3  patients  with  HH  with  SN  have  been  screened  for  the  main  pilot  study 
of  which  1  met  the  criteria  and  was  enrolled  and  completed  the  study. 

Enrollment  for  each  protocol  since  the  start  of  the  study: 

Table  1:  Screened  met  the  study  criteria  and_  enroUed_  in  the  main  pilot  study,  as  of 02/28/2013 _ 


Schepens  IRB 
protocol  number 

HRPO  Log 
Number 

Hemianopia 
Without  neglect 

Hemianopia 
With  neglect 

Normal  vision 

2011-07 

(Prism  Training) 

A-16638.1 

13 

1 

0 

2011-09 

(Eye  movements 
outdoors) 

A-16638.2 

7 

1 

0 

2011-010 
(Virtual  walking 
sim) 

A-16638.3 

9 

1 

0 

2011-011 
(Driving  sim) 

A- 1663 8. 4 

11 

1 

0 

Table  2:  Screened,  but  did  not  meet  the  study  criteria,  as  of 02/28/2013 

(These  subjects  do  not  count  toward  the  total  number  approved) 

The  main  reason  for  not  meeting  the  study  criteria  was  incomplete  hemianopia.  Some  of  the  patients  who  did 
not  meet  the  criteria,  but  still  had  substantial  visual  field  loss,  were  invited  to  participate  in  pre-pilot  work  when 
we  were  refining  our  methodology  in  preparation  and  refinement  of  the  tools  for  the  main  pilot  study. 


Schepens  IRB 
protocol  number 

HRPO  Log 
Number 

Particij 

pre-] 

Without 

neglect 

pated  in 
nlots 

With 

neglect 

Did  not  ] 

Without 

neglect 

participate 

With 

neglect 

2011-07 

(Prism  Training) 

A-16638.1 

5 

1 

3 

1 

2011-09 

(Eye  movements 
outdoors) 

A-16638.2 

0 

0 

0 

0 

6 


2011-010 
(Virtual  walking 
sim) 

A-16638.3 

4 

2 

0 

0 

2011-011 
(Driving  sim) 

A-16638.4 

3 

2 

0 

0 

Preliminary  data 

Below  we  report  preliminary  data  for  each  of  the  main  aspects  of  the  pilot  study. 

A.  Perceptual  Motor  Training 

This  training  involves,  while  wearing  the  prism  glasses,  reaching  out  and  touching  (on  a  touch  screen) 
peripheral  objects  displayed  on  the  blind/neglected  side  and  on  the  seeing  side  of  the  visual  field  while 
maintaining  fixation  on  a  central  target  (Figure  1). 

In  the  last  12  months,  7  participants  with  hemianopia  without  neglect  have  completed  the  full  training  schedule, 
with  significant  improvements  in  the  ability  to  accurately  touch  targets  presented  in  areas  of  prism-expanded 
vision  in  the  blind  hemifield.  Before  training,  the  error  in  touching  was  17°,  which  reduced  significantly  (p  = 
0.001)  to  only  1.0°  after  training  (equivalent  to  seeing  side  accuracy).  Furthermore,  the  difference  between 
seeing  and  prism  side  reaction  times  reduced  significantly  (p=0.04)  from  876ms  to  1 10ms.  These  results  are 
encouraging. 


(c) 


Figure  1:  Computerized  perceptual  motor  training 

(a)  The  training  station  includes  a  padded  antimicrobial  forehead  rest,  wide  screen  touch 
monitor,  fixation  monitoring  camera,  and  operator’s  station,  (b)  Patient  reaches  out  to  touch  the 
peripheral  target  (black  and  white  checkerboard  square  won  a  natural  driving  scene 
background  while  fixating  the  central  pink  cross,  (c)  Close  up  of  patient  in  the  forehead  rest 
wearing  peripheral  prism  glasses. 


B.  Driving  simulator 

Each  driving  simulator  assessment  comprises  five  test  drives  (each  about  10  minutes)  on  pre-determined  routes 
guided  by  computer-generated,  spoken  navigation  cues  (similar  to  GPS  instructions)  (Figure  2a).  While  driving, 
the  participant’s  primary  task  is  to  press  the  horn  button  whenever  he/she  detects  a  pedestrian  figure  that 
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appears  periodically  at  small  and  large  eccentricities  on  the  right  and  left  of  the  roadway  (Figure  2b).  The 
pedestrian  figures  move  with  biological  motion  toward  the  road  on  a  collision  course,  but  do  not  enter  the  travel 
lane.  The  driving  is  highly  engaging  as  there  is  other  traffic  on  the  roads  and  the  participants  have  to  obey  all 
the  normal  rules  of  the  road.  Main  outcome  measures  are  detection  rates  and  reaction  times. 


Figure  2:  (a)  The  driving  simulator  has  5  large  LCD  screens  with  225°  horizontal  field  of  view  and  all  the  controls  in  an 
automatic  transmission  car.  (b)  Screenshot  showing  a  pedestrian  walking  toward  the  road  as  the  participant’s  vehicle 
approaches  (central  monitor  only). 


Preliminary  results  for  9  participants  with  hemianopia  without  neglect  suggest  a  significant  improvement  in 
detection  of  blind  side  pedestrian  hazards  with  the  prism  glasses  following  training.  Median  blindside  detection 
rates  for  approaching  pedestrian  hazards  improved  from  63%  without  prism  glasses  to  88%  with  prism  glasses 
(p  =  0.011;  Figure  3). 


Percent  detected  without  prisms 


Figure  3:  Detection  rates  for  blind  side  pedestrian  hazards 
without  prism  glasses  before  training  and  with  prism 
glasses  after  training.  On  average,  detection  rates  were 
better  with  than  without  prism  glasses  (points  all  on  or 
above  the  diagonal). 
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C.  Collision  judgments  in  virtual  mall  walking  simulator 

The  virtual  mall  is  a  virtual  reality  model  of  a  real  shopping  mall  (Figure  4).  Participants  are  lm  in  front  of  a 
large  rear-projection  screen  on  which  the  shopping  mall  is  displayed.  Their  task  is  to  report  whether  they  would 
collide  with  stationary  obstacles  (life-sized  human  figures;  Figure  4)  that  periodically  appear  at  different  offsets 
(up  to  120cm  to  each  side)  from  the  simulated  participant’s  walking  direction.  Outcome  measures  are  detection 
rates  for  obstacles,  the  perceived  safe  passing  distance  on  the  blind  and  seeing  sides,  and  judgment  uncertainty. 


Figure  4.  Illustration  of  the  virtual  reality  mall  set-up  and  collision  judgment  task. 

Participants  are  instructed  to  fixate  a  cross  at  the  center  of  the  screen  (green  x)  and  imagine  that  they  are  walking  in  a 
real  shopping  mall.  A  figure  appears,  moves  in  a  linear  path,  and  disappears  after  1  second.  The  participant  clicks  a 
button  when  they  detect  the  figure  and  then  responds  verbally  “collision”,  “no  collision”,  or  “nothing”  for  each  trial.  The 
figure  may  appear  at  any  eccentricity  offset  to  the  right  or  left  of  center,  or  not  at  all  (1 0%  of  trials).  This  is  repeated  for  a 
total  of  88  trials. 


For  conditions  in  which  participants  fixated  a  central  cross  on  the  screen,  blindside  detection  rates  were 
significantly  better  with  than  without  the  prism  glasses  (improved  by  59%  with  prism  glasses,  p  =  0.01).  In 
addition,  under  natural  scanning  conditions,  detection  rates  also  increased  with  the  prism  glasses  (by  1 8%)  to  a 
median  of  100%.  When  wearing  the  prism  glasses,  detection  rates  on  the  blind  side  were  as  good  as  those  on  the 
seeing  side.  Collision  judgments  for  pedestrians  in  the  blind  hemifield  made  from  the  prism-expanded  vision 
differed  from  collision  judgments  on  the  seeing  side.  As  the  data  for  judgments  in  the  blind  hemifield  cannot  be 
adequately  fit  by  a  traditional  psychometric  function  (unlike  seeing  side  judgments),  we  are  currently 
developing  an  algorithm  to  quantify  blind  hemifield  collision  judgments  when  using  the  prism  glasses. 

D.  Eye  and  head  movements  when  walking  outdoors 

Using  the  integrated  eye  and  head  tracking  system  (Figure  5),  we  have  collected  and  processed  data  for  7 
subjects  with  hemianopia  without  neglect.  Each  participant  walked  short  outdoor  routes  along  busy  downtown 
Boston  streets.  Figure  6a  shows  an  example  of  the  output  for  the  gaze  tracking  of  a  right  hemianope  during  a  17 
s  period  from  a  walk  without  prism  glasses.  To  derive  the  gaze  position  (red  line  in  Figure  6),  the  eye  movement 
data  were  synchronized  and  added  to  the  head  position  information  (blue  line  in  Figure  6). 

We  have  conducted  a  preliminary  analysis  of  the  gaze  data  from  two  visits  for  each  subject,  including  with-  and 
without-  prism  walks  at  each  visit.  Gaze  movements  were  analyzed  within  the  blind  and  seeing  hemifields  as 
the  sum  of  the  gaze  movement  amplitudes  divided  by  the  total  number  of  samples.  Patients  had  a  trend  to  bias 
their  gaze  position  towards  the  blind  side  when  wearing  the  prisms  but  not  when  walking  without  them 
(significant  side*prism  interaction,  p=0.009).  This  suggests  that,  due  to  the  prisms,  the  patient’s  attention  was 
attracted  to  their  blind  side  (as  indicated  by  the  gaze  direction).  As  yet  we  have  insufficient  data  to  evaluate  the 
effects  of  training  on  gaze  behaviors. 
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Figure  5:  (a)  The  mobile  gaze  tracking  system  including  two  motion  sensors  to  measure  head  position 
relative  to  body  position,  the  Positive  Science  eye  tracker  and  two  small  notebook  computers  for  data 
recording,  (b)  Close  up  of  the  eye  tracker  eye  and  scene  cameras  mounted  on  a  custom  frame  suitable 
for  the  mounting  of  prism  lenses. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


In  months  12  to  24,  we  have: 

•  Renewed  all  necessary  IRB  approvals; 

•  Made  excellent  progress  in  recruitment  and  running  participants  with  HH  without  SN  and  are  nicely  on 
track  to  have  12  participants  in  this  group  complete  the  study; 

•  Made  much  progress  in  the  head  and  eye  tracking  aspects  of  the  project,  in  particular  for  the  outdoor 
walking  task. 


REPORTABLE  OUTCOMES 

•  Preliminary  data  for  the  virtual-mall  walking-simulator  collision  detection  task  was  presented  at  the 
American  Academy  of  Optometry  2012  meeting  in  October. 

•  An  abstract  reporting  preliminary  data  from  the  prism  training  and  pre-  and  post-training  results  from  the 
virtual-mall  walking- simulator  collision  detection  task  has  been  accepted  for  a  poster  presentation  at  ARVO 
2013  (Association  for  Research  in  Vision  and  Ophthalmology). 

•  An  abstract  reporting  preliminary  data  from  the  outdoor  walking  task  has  been  accepted  for  a  poster 
presentation  at  ARVO  2013 

•  Dr  Houston  gave  an  invited  talk  at  the  New  England  College  of  Optometry  about  the  methodology  and  early 
results  from  the  study 

•  Dr  Luo’s  postdoctoral  fellow  gave  a  talk  at  Schepens  about  the  development  of  the  outdoor  eye  and  head 
tracking  methodology  and  preliminary  results. 

•  Three  papers  are  in  preparation:  a  paper  addressing  the  development  of  the  prism-training  methodology,  a 
paper  reporting  data  from  the  collision  judgment  task  in  the  virtual-mall  walking-simulator,  and  a  paper 
addressing  the  methodology  for  integrating  eye  and  head  movement  data  from  the  outdoor  walking  task. 


CONCLUSIONS 

In  this  second  year  we  have  made  excellent  progress  in  recruiting  and  running  participants  with  hemianopia 
without  neglect.  We  are  clearly  on  track  to  reach  our  goal  of  12  participants  with  hemianopia  without  neglect  to 
complete  the  whole  study,  with  5  already  completed  and  7  currently  part  way  through  the  protocol.  We  have 
also  established  routes  of  referral  for  patients  with  hemianopia  with  neglect  and  will  start  to  increase 
recruitment  of  these  patients  when  the  cohort  without  neglect  has  completed  all  training  visits. 
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